Add Gemma 3n support (E2B, macOS) by stikves · Pull Request #198 · apple/coreai-models

stikves · 2026-08-26T01:37:50Z

Summary

Add Google Gemma 3n E2B for macOS (4-bit quantized)
Novel on-device architecture with AltUp, LAUREL, KV sharing, Gaussian TopK sparsity, dual RoPE, QKV norms
12 unit tests covering parity, KV sharing, sparsity, float16, AltUp, and tied embeddings

Evaluation

Compression	Wikitext PPL	tinyMMLU acc
none (float16)	61.53	58.75%
4-bit quantized	72.13	59.27%

High wikitext PPL is expected for instruction-tuned models — tinyMMLU accuracy is the appropriate quality metric.

Test plan

12 unit tests pass (parity, KV sharing, AltUp, sparsity, float16)
Export succeeds: uv run coreai.llm.export google/gemma-3n-E2B-it
Inference produces coherent text

Add Google Gemma 3n E2B — a novel on-device architecture with 7 features not found in other supported models: - AltUp: 4-copy hidden state with learned predict/correct routing - LAUREL: low-rank residual branch (rank 64) per layer - Per-layer input embeddings with gated injection - KV cache sharing: last 10/30 layers reuse K/V (33% cache savings) - Gaussian TopK: 95% activation sparsity in first 10 MLP layers - Dual RoPE: local (theta=10K, window 512) + global (theta=1M) - QKV norms: RMSNorm on Q/K (with scale), V (without scale) Evaluation: WikiText PPL 61.53 (float16), tinyMMLU 58.75%. High wikitext PPL is expected for instruction-tuned models; tinyMMLU accuracy is the appropriate quality metric.

tjia1818 · 2026-08-28T05:01:41Z

+
+| Compression      | Wikitext PPL | tinyMMLU acc |
+| ---------------- | ------------ | ------------ |
+| none (float16)   | 61.53        | 58.75%       |


let's align this table with other models (I don't think we include tinyMMLU numbers for others).

Table columns/names also needs to be aligned

stikves force-pushed the sukru/gemma3n branch 3 times, most recently from c36cf23 to e389279 Compare August 26, 2026 01:54

stikves self-assigned this Aug 26, 2026

stikves force-pushed the sukru/gemma3n branch from e389279 to 2b3d111 Compare August 26, 2026 02:18